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after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 
• Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
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1 )£3 Responsive to communication(s) filed on 19 March 2004 . 
2a)D This action is FINAL. 2b)^ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 
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4) ^ Claim(s) 1-42 is/are pending in the application. 
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Application Papers 
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1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 
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2.D Certified copies of the priority documents have been received in Application No. . 
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application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 
Election/Restriction 

1. Applicants election was made without traverse in Paper No. 7, filed March19, 
2004 is acknowledged. Applicant elected group I, claims 1-11, 15-26, 41 and 42. 

2. Claims 12-14, 27- 40 are withdrawn from further consideration pursuant to 37 
CFR 1.142(b) as being drawn to a nonelected invention, there being no allowable 
generic or linking claim. 

Drawings 

3. The drawings are objected to because of draftperson's remarks (see attached 
PTO-948 paper number 8. A proposed drawing correction or corrected drawings are 
required in reply to the Office action to avoid abandonment of the application. The 
objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 U.8.C. §102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless 

(e) the invention was described in (L) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before the 
invention by the applicant for patent, except that an international application filed under the 
treaty defined in section 351(a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 
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Claims 1-6, 15,16, 41 and 42 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Sundareswaran et al. (US. 6,330,356 B1). 

As to claim 1 , Sundareswaran discloses a method for accurately estimating a 
pose of a camera (fig 1 , 20) within a scene using a three-dimension model of the scene 
(fig 1, 30, column 6, lines 7-14, 19- 24), comprising the steps of: 

(a) generating an initial estimate of the pose (column 6, lines 19-42); 

(b) selecting a set of relevant features (ring shaped marker corresponds to 
features, fig 3, 72) of the three dimensional model based on the initial estimate of the 
pose (column 6, lines 37- 48); 

(c) creating a virtual projection (column 4, lines 12- 21) of the set of relevant 
features responsive to the initial estimate of the pose (column 6, lines 48- 58, column 7, 
lines 2-58); 

(d) matching a plurality of features of an image received from the camera to the 
virtual projection of the set of relevant features and measuring a plurality of matching 
errors (computing a disparity between the positions of the features and 
corresponding positions of features in a calculated projection of a stored, 
three-dimensional object model CAD, and calculated projection based on a virtual 
camera model, column 8, lines 56- 67, column 9, linel to column 10, lines 1- 56); and 

(e) updating (based on the position and orientation parameters of the virtual 
camera are identified by rotating and shifting the virtual camera point of view, column 
1 1 , lines 2-7) the estimate of the pose to reduce the plurality of matching errors (fig 1 
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and 7, column 6, lines 60- 67, column 8, lines 56-67, column 9, line 1 to column 10, 
lines 1-56). 

As to claim 15, see the rejection of claim 1. 

As to claim 41 , see the rejection of claim 1 . 

As to claim 42, claim 42 recites similar limitation as claim 1 above and similarly 
analyzed except for the step (f) point of view refinement means for updating the 
estimate of the point of view to reduce the plurality of matching errors (column 8, line 
56- 67, column 9, lines 1- 45, column 10, lines 52- 56); and 

(g) model refinement means, responsive to the estimated point of view and to 
the image, for updating the three-dimensional model as taught by Sundareswaran 
(column 8, line 56- 67, column 9, lines 1- 45, column 10, lines 52- 56). 

As to claim 2, Sundareswaran discloses the method further comprising the step 

of: 

(f) repeating to steps (c), (d), and (e) using the updated estimate of the pose until 
the plurality of matching errors are less than predetermined matching criteria (fig 1 and 
7, column 6, lines 60- 67, column 8, lines 56-67, column 9, line 1 to column 10, lines 1- 
56). 

As to claim 3, Sundareswaran discloses the method further comprising the steps 

of: 

(f) adding the updates to the estimate of the pose made in step (e) to a total 
change value of the estimate of the pose (fig 1 and 7, column 6, lines 60- 67, column 8, 
lines 56-67, column 9, line 1 to column 10, lines 1-56). 
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(g) if the total change value is less than predetermined (column 7, lines 32- 35, 
column 9, lines 7-38) pose change criteria (column 4, lines 12- 21), resetting the total 
change value and repeating to steps (b), (c), (d), (e) and (f) using the updated estimate 
of the pose (column 6, lines 60- 67, column 11, lines 2-7); and 

(h) repeating to steps (c), (d), (e), (f) and (g) using the updated estimate of the 
pose until the plurality of matching errors to are less than predetermined matching 
criteria (fig 1 and 7, column 6, lines 60- 67, column 8, lines 56-67, column 9, line 1 to 
column 10, lines 1-56). 

As to claim 4, Sundareswaran discloses the method wherein step (a) includes 
the step of comparing an image received from the camera to the three-dimensional 
model of the scene to generate the estimate of the pose (column 1 1 , lines 17-58). 

As to claim 5, Sundareswaran discloses the method wherein step (a) includes 
the step of using a preceding pose estimate of a preceding image received from the 
camera to generate the estimate of the pose (column 6, lines 60- 67, column 8, lines 56- 
67). 

As to claim 6, Sundareswaran discloses the method wherein step (b) includes 
the steps of: 

b1) Z-buffering the three dimensional model based on the estimated pose to 
determine a set of visible features of the three dimensional model (column 4, lines 13- 
36, column 11, lines 17-67, column 12, lines 1-5); and 

(b2) selecting the set of relevant features from the set of visible features (column 
4, lines 13-36). 
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As to claim 16, Sundareswaran discloses the method wherein step (b) includes 
the step of: 

(bl) projecting a plurality of color values of the image onto the three dimension 
model of the scene to update a texture of a surface of the three dimensional model 
(column 6, lines 19- 25). 

Claim Rejections - 35 U.S.C §102 
5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless -- 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use 
or on sale in this country, more than one year prior to the date of application for patent in the United States. 

Claims 17 and 18 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Cox et al. (US. 5,644,651). 

As to claim 17, Cox discloses a method for accurately estimating a position of 
remote vehicle using (column 6, lines 21-31 ) a three-dimension model and an image 
from a camera having a known orientation relative to the remote vehicle (column 1, lines 
62- 67, column 2, lines 1-31), comprising the steps of: 

(a) comparing the image to the three dimension (column 1 , lines 8-12) model of 
the scene to generate an estimate of the pose (column 2, lines 24-53, column 4, lines 
53-60); 

(b) selecting a set of relevant features (epipolar lines are considered to be a set 
of features, column 3, lines 8-22) of the three dimensional model based on the estimate 
of the pose (column 5, lines 7- 42); 
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(c) matching (column 4, lines 12-67) a plurality of features of the image to the set 
of relevant features and measuring a plurality of matching errors (column 5, lines 7- 42); 

(d) updating the estimate of the pose based on the plurality of matching errors 
(column 5, lines 1-67, column 6, lines 1-31); and 

(e) determining the position of the remote vehicle based on the estimate of the 
pose and the orientation of the camera (column 6, lines 21-31). 

As to claim 18, Cox discloses the method of claim 17, further comprising the step 

of: 

(f) updating the three dimensional model based on data from the image and the 
updated estimate of the pose (column 5, lines 1-67, column 6, lines 1-31). 

Allowable Subject Matter 
6. Claims 7-11, are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 
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Reason For Allowance 
7. Regarding claim 19, The prior art of record fails to teach or suggest, alone or in 
combination, a method for refining a three dimensional model of a scene containing an 
object using a plurality of images of the scene, each image including the object, 
comprising the step of: (d) matching a plurality of first features of the first image to the 
first set of relevant features and measuring a plurality of first matching errors; 

(f) matching a plurality of second features of the second image to the second set 
of relevant features and measuring a plurality of second matching errors. 

Claim 19-23 are allowed. 

Regarding claim 24, The prior art of record fails to teach or suggest, alone or in 
combination, a method for refining a three dimensional model of a scene containing an 
object using a plurality of images of the scene, each image including the object, 
comprising the step of: (a) selecting a subset of images from the plurality of images of 
the scene, the subset of frames containing at least two of the images; 

(c) comparing each image in the subset of images to the three dimensional 
model to generate a subset of estimated viewpoints corresponding to the subset of 
images, the subset of estimated viewpoints constrained by the plurality of approximate 
relative viewpoints; 

(e) matching a plurality of features of the each image in the subset of images to 
the corresponding set of relevant features and measuring a plurality of matching errors. 
Claim 24 - 26 are allowed. 
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Other prior art cited 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Anandan et al. (US.6,192,145B1) discloses method and apparatus for three- 
dimensional scene processing using parallax geometry of pairs of points. 

Kumar et al. (US.6,522,787B1) discloses method and system for rendering and 
combining images to form a synthesized view of a scene containing image information 
from a second image. 

Imaizumi et al.(US. 6,047,087) discloses image processor. 

Martin et al. (US. 5,850,469) discloses real time tracking of camera pose. 
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Contact Information 



9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sheela C Chawan whose telephone number is 703- 
305- 4876. The examiner can normally be reached on Monday - Thursday 6-7. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on 703-308-5246. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Sheela Chawan 
Patent Examiner 
Group Art Unit 2625 
April 2, 2004 
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